This article originally appeared in The Bar Examiner print edition, Winter 2023-2024 (Vol. 92, No. 4), pp. 1-2.
By John J. McAlaryAs I write this letter, our staff in New York State just completed administering the February bar examination. This was the 52nd exam that I have administered since beginning work in bar admissions in 1998, but it never seems to get easier given the challenges of administering the exam to thousands of applicants at several locations. We examined about 4,000 applicants from 164 US law schools; examinees came from 48 US states and territories and more than 100 foreign countries. The exam was held in four cities across New York, as well as at two sites for accommodated applicants.
For readers unfamiliar with the logistics of administering a high-stakes licensing examination, administrators must engage in careful advance planning to project the number of applicants who will apply for each exam, procure adequate facilities for holding the exam while often navigating stringent state requirements, employ proctors, rent tables and chairs, retain security, wire the tables to provide electrical for the applicants using laptops for the Multistate Essay Examination (MEE) and Multistate Performance Test (MPT), and arrange for the secure and timely transfer of exam materials between our office and the test sites. After the conclusion of each exam session, we must collect and account for all examination booklets before dismissing the applicants. Exams then need to be boxed up and securely returned to our office in Albany.
Exam days can be gruelingly long, as we arrive by 6:30 a.m. and often do not leave the site until after 6:00 p.m. We can rack up 10,000–20,000 steps per day. Any number of issues may occur before, during, and/or after the exam that could affect the administration. Administrators must be adept at contingency planning and train their staff and proctors on what to do in the event of: bad weather that may disrupt the delivery of exam materials or the exam itself, absent proctors, an HVAC issue or a power outage at a test site, medical emergencies involving an examinee or a proctor, software issues, and cheating or other misconduct. Quick and decisive action is often required, and having a list of steps to take in such situations is a necessity. If all goes as planned, we are rewarded with a boring and uneventful exam.
There is no time to rest when we return to the office, as the days following the exam are often busier than the week prior. Exam boxes need to be unpacked; Multistate Bar Examination answer sheets must be scanned before shipping them to NCBE for grading; handwritten exams must be separated, scanned, and uploaded to our grader site; MEE and MPT exam file uploads must be reconciled and then exported from the vendor site to our grader site; attendance records must be checked and confirmed; irregularity reports prepared; and graders need to be sent materials for the grader calibration sessions that are held the week after the exam. Only after the graders are set to start grading can we take a step back, exhale, and relax—but only briefly until we start the whole process over again for the next exam. There is both relief and a great sense of accomplishment at the conclusion of each exam cycle.
Since my previous column there have been some new developments regarding the NextGen bar exam. As of this issue’s publication, 17 jurisdictions have adopted the NextGen bar exam. In January, NCBE announced that Surpass Assessment will provide the testing software for the new exam. NCBE also publicized its collaboration with the AccessLex Institute in the development of NextGen study aids.
NCBE conducted a nationwide field test for the NextGen bar exam in January as well. More than 4,000 students from 88 law schools took the field test in 41 US jurisdictions. The field test permitted the exam developers and psychometricians at NCBE to gather data to assess the performance of the new question types and to confirm the timing allotments/assignments for answering each of them, with the goal of making the new exam as fair and equitable as possible. The field test also allowed NCBE to test computerized exam delivery and provided graders an opportunity to practice grading the new question items. NCBE used experienced bar exam graders from among the jurisdictions and provided training so the graders were well prepared to practice and provide feedback on the grading process. This valuable data will assist NCBE in ensuring that the NextGen bar exam assesses the knowledge and skills that new lawyers need fairly and accurately.
I encourage you to take the time to read through this latest issue of the Bar Examiner, which includes articles on relevant topics including the benefits of military spouse rules and recent changes by the ABA regarding distance education and academic freedom on law school campuses. This issue also contains tributes to Mark Albanese, PhD, NCBE’s former Director of Testing and Research, who passed away in late December. I had the pleasure of knowing Mark and worked with him on several projects over the years. I enjoyed sitting next to Mark at meals during conferences because I learned so much about scaling and scoring during our conversations. I will miss our walks after the conclusion of conferences. Mark was intelligent and sharp, and he had a dry sense of humor that never failed to bring a smile to your face when he spoke about potentially tedious topics such as testing and measurement. We will miss him dearly.
Finally, if you are new to the bar admissions community, I encourage you to become knowledgeable about and involved in NCBE activities. One of the best ways to learn about the organization and bar admissions is to attend the NCBE Annual Bar Admissions Conference in the spring and to consider serving on one of the many NCBE committees. The NCBE staff and the Education Committee always put together an outstanding program at the annual conference, and this year’s program in Chicago is exceptional. I am confident you will leave the conference knowing more about bar admissions than you did when you arrived.
Kindest regards,
John J. McAlary
Contact us to request a pdf file of the original article as it appeared in the print edition.
